System and method for processing multiple loose gemstones using image-based analysis techniques

ABSTRACT

Systems and methods for processing a plurality of loose gemstones resting on a worksurface are disclosed. The system comprises an image capturing device for capturing an image of the plurality of loose gemstones on the worksurface. The system further comprises a computing device having a non-transitory computer-readable storage medium and a processor. Additionally, the processor is configured by executing a software application to analyze the image to detect edges of objects within the image and generate a diagram representing the detected edges of the gemstones. The processor is further configured to detect, using a trained neural network processing the image and the diagram, individual gemstones within the image and count the individual gemstones detected within the image to determine a gemstone count. The processor is also configured to perform, using an output device, a processing operation on the gemstones as a function of the gemstone count.

FIELD OF THE INVENTION

The invention relates to systems and methods for processing gemstones,and in particular, systems and methods for processing large quantitiesof gemstones using image-based analysis techniques.

BACKGROUND OF THE INVENTION

Jewelry and gemstone merchants often deal in relatively large quantitiesof loose gemstones on the order of tens, hundreds and even thousands ofunits. Processing multiple loose gemstones is often performed manually.As an example, processing can include counting, sorting, categorizing,evaluating and selecting gemstones, say, for inclusion in a jewelryitem. Manual processing is time consuming and labor intensive. Existingsystems for processing gemstones using computer vision systemsimplementing image-based analysis techniques can lack reliability whenprocessing images depicting relatively large quantities of loosegemstones.

It is in regard to these and other problems in the art that the presentdisclosure is directed to provide a technical solution for an effectivesystem and method for automated processing of multiple loose gemstonesusing image-based analysis techniques.

SUMMARY OF THE INVENTION

According to an aspect of the present disclosure, a method forprocessing a plurality of loose gemstones resting on a worksurface isdisclosed. The method is implemented by a computing device having anon-transitory computer-readable storage medium and a processorconfigured by executing a software program stored in the storage medium.The method comprises the step of receiving by the processor, from animage capturing device, an image of the plurality of loose gemstones onthe worksurface. The method further includes the step of analyzing, bythe processor, the image to detect edges of objects within the image andgenerating a diagram representing the detected edges of the gemstones.The method also includes the step of detecting individual gemstoneswithin the image by the processor using a trained neural networkprocessing the image and the diagram and counting the individualgemstones detected within the image to determine a gemstone count.Additionally, the method includes the step of performing, by theprocessor, a processing operation on the gemstones as a function of thegemstone count.

According to a further aspect of the disclosure a system for processinga plurality of loose gemstones resting on a worksurface. The systemcomprises an image capturing device for capturing an image of theplurality of loose gemstones on the worksurface. The system furthercomprises a computing device having a non-transitory computer-readablestorage medium and a processor. Additionally, the processor isconfigured by executing a software application to analyze the image todetect edges of objects within the image and generate a diagramrepresenting the detected edges of the gemstones. The processor isfurther configured to detect, using a trained neural network processingthe image and the diagram, individual gemstones within the image andcount the individual gemstones detected within the image to determine agemstone count. The processor is also configured to perform, using anoutput device, a processing operation on the gemstones as a function ofthe gemstone count.

It is with respect to these and other considerations that the disclosuremade herein is presented.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a block diagram of a system embodying the teachingsof the present invention;

FIG. 2 illustrates a flowchart showing steps in an exemplary method forverification of a jewelry item;

FIG. 3 illustrates a flowchart showing steps for online appraisal of thejewelry item;

FIG. 4 illustrates the image of the jewelry item having a gemstone;

FIG. 5 illustrates an exemplary diamond report of Gemological Instituteof America;

FIG. 6 illustrates a line diagram of the jewelry item;

FIG. 7a illustrates a line diagram of the jewelry item at a timeinterval T1;

FIG. 7b illustrates a line diagram of the jewelry item at a timeinterval T1;

FIG. 7c shows the superimposed image of FIG. 7a and FIG. 7 b;

FIG. 8a illustrates a line diagram of the jewelry item with gemstoneremoved from the prongs at time interval T1;

FIG. 8b illustrates a line diagram of the jewelry item at time intervalT2 after the gemstone is set between the prongs;

FIGS. 9a and 9b illustrates the line diagrams of the jewelry item withdifferent prong size at time interval T1 and T2, respectively;

FIG. 10 shows an image of a ring, a gemstone and a standard size coin;

FIG. 11a illustrates the top view of the jewelry item with an initialset gemstone;

FIG. 11b illustrates the top view of the jewelry item with the samegemstone reset in the jewelry item;

FIGS. 12a, 12b, 12c, 12d, 12e, 12f, 12g and 12h illustrates an exemplarymobile application of an online commerce platform for buying and sellingthe jewelry items;

FIG. 13A is a photograph of part of a gem's girdle captured using adigital camera from a top-side perspective view in accordance with anembodiment;

FIG. 13B is a photograph of part of a gem's girdle captured using adigital camera from a bottom-side perspective view in accordance with anembodiment;

FIG. 13C is an exemplary line to line and point to point diagram ofgemstone features including the girdle of the gemstone shown in FIG. 13Agenerated in accordance with an embodiment;

FIG. 13D is an exemplary line to line and point to point diagram ofgemstone features including the girdle of the gemstone shown in FIG. 13Bgenerated in accordance with an embodiment;

FIG. 14A is an image of a rough diamond captured after being split intotwo halves in accordance with an embodiment; and

FIG. 14B is a point to point and line to line diagram of detectedfeatures of the outer-surface and the edges of two halves of a splitstone overlaid on the image of FIG. 14A in accordance with anembodiment;

FIG. 15 is a block diagram of the exemplary system shown in FIG. 1including a light emitter and rotating/tilting gemstone mount inaccordance with an embodiment;

FIG. 16 is a block diagram of an exemplary system for processingmultiple loose gemstones using image-based analysis techniques inaccordance with an embodiment;

FIG. 17 is a process flow diagram of an exemplary method for processingmultiple loose gemstones using image-based analysis techniques inaccordance with an embodiment;

FIG. 18 is a screen shot of an exemplary display output of the system ofFIG. 16 showing an image of multiple loose gemstones on a worksurfaceaugmented with markings in accordance with one or more embodiments; and

FIG. 19 is a top plan view of an exemplary worksurface of the system ofFIG. 16 with reference markings thereon in accordance with one or moreembodiments.

DETAILED DESCRIPTION

While the disclosed subject matter is amenable to various modificationsand alternative forms, specifics thereof have been shown by way ofexample in the drawings and will be described in detail. It should beunderstood, however, that the intention is not to limit aspects of theinvention to the particular embodiments described. The intention is tocover all modifications, equivalents, and alternatives falling withinthe spirit and scope of the invention.

The following detailed description is made with reference to thefigures. Exemplary embodiments are described to illustrate the subjectmatter of the disclosure, not to limit its scope, which is defined bythe claims. Those skilled in the art will recognize that the variouselements described and/or shown may be arranged in various combinationsand configurations without departing from the scope of the disclosure.

All numbers or values are herein assumed to be modified by the term“about.” The disclosure of numerical ranges by endpoints includes allnumbers subsumed within that range (e.g., 1 to 5 includes 1, 1.5, 2,2.75, 3, 3.80, 4, and 5).

As used in this specification and the appended claims, the singularindefinite articles “a”, “an”, and the definite article “the” should beconsidered to include or otherwise cover both single and pluralreferents unless the content clearly dictates otherwise. In other words,these articles are applicable to one or more referents. As used in thisspecification and the appended claims, the term “or” is generallyemployed to include or otherwise cover “and/or” unless the contentclearly dictates otherwise.

FIG. 1 illustrates a block diagram of a system 100 embodying theteachings of the present invention. The system 100 includes an imagecapturing device 102 for capturing multiple images of a jewelry item.The image capturing device 102 takes images from various angles of thejewelry item including the gemstones, metal parts and the prongs. Thephysical characteristics of the jewelry item including the appearance,shape, size, measurements (length, width and depth) and color arecaptured through the images. The captured images can be in form of stillimages, video images or laser based topography model of the jewelryitem. Exemplary image capturing device includes an analog or digitalstill image camera, a video camera, an optical camera, a laser camera, alaser or 3D image scanner, or any other device capable of capturing highresolution images of the jewelry item. The image capturing device 102can also be a high definition inbuilt camera of a communication devicesuch as a computer, a laptop or a mobile phone.

In an exemplary embodiment, the image capturing device can take multipleimages of the jewelry item and stitch the images to get a 360° view.Alternatively, the 360° view can be captured using the panoramic viewtechnology of the image capturing device 102. In a further alternativeembodiment, the video image of the jewelry item is captured and stillimages are extracted from the video frames. FIG. 4 illustrates an imageof a jewelry item 400 having a gemstone 402. The image capturing device102 takes 3D images of the metal part 406, the gemstone 402 and theprongs 404.

The image capturing device 102 also captures the image of inclusions inthe gemstone of the jewelry item. Examples of inclusions which may becaptured include one or more of solid, liquid or gas inclusions,internal fissures, fractures, zones of crystal growth, stress cracks,bubbles, cleavage fault, feather, halo inclusions, two-phase andthree-phase inclusions, etc. The image capturing device 102 furthercaptures identification markings or inscriptions on the surface of themetal part or the gemstone of the jewelry item. Any nicks, scratches,dents, placements, table facets, bottom facets of gemstone, color changeunder lighting conditions, girdle facets and thickness are also capturedthrough images.

The captured images through image capturing device 102 are stored in astorage module 104. The storage module 104 also stores a time stampcomprising the date and time and a location of capturing and storing theimages. The storage module 104 can be an inbuilt storage memory of theimage capturing device 102, for example the internal memory of a cameradevice. In an alternative embodiment, the storage module 104 can bestorage memory of one or more of a server computer, a client usercomputer, a personal computer (PC), a tablet PC, a laptop computer, adesktop computer, a mobile phone, a control system and a network router,switch or bridge. Alternatively, the storage module 104 can be cloudstorage of a virtual cloud environment.

The stored images are provided to a processing unit 106 for extractingthe information from images and processing it for security analysis ofthe jewelry item. A user (herein ‘user’ refers to a jewelry owner,manufacturer, lab person, etc. and will be considered for thedescription below) can provide additional information of the jewelryitem to the processing unit 106 through an input system 108. Theadditional information can be related to physical characteristics of themetal part and the gemstone like weight, volume, dimensions, carat, cutgrade, color rating, etc. The user can also input information relatingto price, labor, etc. of the jewelry item. A certificate or lab reportof the jewelry item can also be entered manually or uploaded though theinput system 108. FIG. 5 shows an exemplary diamond report ofGemological Institute of America. The report shows the physicalcharacteristics of the diamond along with an image. Examples of inputsystem 108 include a keyboard, a touch panel, an electronic or opticalmouse, a gesture recognition system and a voice input system. The inputsystem 108 provides the additional information through a user interfaceof software, a website or a mobile application.

The processing unit 106 extracts information from the jewelry itemimages captured at different time intervals using a software program andcompare the information for detecting any changes in the metal part orgemstones of the jewelry item. The processing unit 106 can be aninternal CPU of the image capturing device 102 like a digital camera. Inan alternative embodiment of the present invention, the processing unit106 can be the processing unit of one or more of a server computer, aclient user computer, a personal computer (PC), a tablet PC, a laptopcomputer, a desktop computer, a mobile phone, a control system and anetwork router, switch or bridge. The processed information from theprocessing unit 106 is stored in the storage module 104 or an externalstorage 112.

The processed information of the jewelry item by the processing unit 106is sent to an output system 110 which displays the information to theuser. The output system 110 can display the information pertaining tothe physical characteristics of the jewelry item including theappearance, shape, size, volume, weight, measurements (length, width anddepth) and color. The information related to inclusions in thegemstones, inscriptions on the metal part and gemstones, nicks,scratches, dents, placements, angles of jewelry and prongs, stonequality, table facets, bottom facets of gem, girdle facets andthickness, weights and or all known and deduced information from thejewelry can be displayed to the user. All the above-mentionedinformation is extracted from the images of jewelry item taken atdifferent time interval for comparison and reference. The time stamprelated to capturing and storing the images are also displayed throughthe output system 100. In a preferred embodiment of the presentinvention, the output system 110 also shows the difference in prongsettings of the jewelry item, holding the gemstones, before sending itfor repairment and after receiving it from repairment. The processingunit 106 calculates the difference in the metal part and the gemstonebased on the changes in the prong settings of the jewelry item. Thedifference in the metal part and the gemstone is also shown to the useras the actual value or a percentage of the difference. The output system110 can also display an alert or a flag if the difference calculated bythe processing unit 106 is more than a predefined threshold limit. Theoutput system 110 can also display the information in form of acertificate including the physical characteristics and images of thejewelry item. The output system 110 can also show the valuation of thejewelry item calculated from the images of different time intervals. Thedifference in valuation is also shown to the user.

The external storage 112, which can be memory storage of a remotecomputer, a laptop, a mobile phone, a network router, a switch, a bridgeor a virtual space in a cloud environment, is connected to an onlinecommerce platform 114. The online commerce platform 114 enables the userto display the jewelry item, evaluate it by comparing with other similarjewelry items from other websites and sell the jewelry item. The onlinecommerce platform 114 displays the physical characteristics and sellingprice of the jewelry item. The user can also select and buy or selljewelry items from the online commerce platform 114 using an appropriatepayment method.

FIG. 2 illustrates a flowchart showing steps for verification of ajewelry item. The process starts at step 202 and a jewelry itemcontaining gemstones held by gem holders (prongs) is taken at step 204.At step 206 the image capturing device 102 takes a first set of imagesof the jewelry item at time interval T1. In an exemplary embodiment ofthe present invention, the time interval T1 is considered as the timebefore the jewelry item is sent to the jeweler for repair. The time T1can also be any arbitrary time at which the images ofjewelry item aretaken and stored for reference in future. The images can also becaptured at regular intervals by the user to keep a track of the wearand tear in the jewelry item with time. The first set of images includesthe images of metal part, gemstones and the prongs of the jewelry item.In case the gemstones are missing or fallen from the jewelry item, theimages of empty prongs are also captured. If the user provides thegemstones separately to the jeweler for fitting on the jewelry item, theimages of gemstones are also captured for comparison with the jewelryitem after it returns from the jeweler. The images are captured at highresolution to enable magnification for easy viewing and comparison. Thecaptured images are stored in the storage module 104.

At step 208, the first set of images are stitched together to get acomplete view of the jewelry item. The images can be stitched by theimage capturing device 102. Alternatively, the images can be stitched bythe processing unit 106. The stitched images are also stored in thestorage module 104. A software algorithm running on the processing unit106 extracts information from the first set of images and separates outthe metal part, gemstones and the prongs of the jewelry item using anedge detection methodology at step 210. The algorithm creates apoint-to-point or line diagram of the complete jewelry item includingprongs and the facet structure at step 212 and separates out the variousparts. The edge detection methodology is well known in the art and willnot be discussed here in detail for the purpose of brevity of thesubject. It should, however, be understood to a person skilled in theart that any other known technology can be used to separate the variousparts of the jewelry item. The algorithm extracts one or more of thefollowing information from the first set of images at step 214:

Metal part dimensions, volume, angle, color and weight

Metal quality/caratage

Nicks, scratches, dents, cracks on the metal part or gemstones

Inclusions in the gemstones

Identification markings or inscriptions on the metal part and gemstones

Table facet structure and girdle structure of gemstones

Angle and height of gemstones

Pavilion depth and angle

Crown height and angle

Weight and color of the gemstones

Coverage of the gemstone surface area, i.e., is it too short (near edge)or too high (towards table) or correct distance

Distance between the prongs

How the prongs are placed in respect to one another and in respect tothe gemstones

Height and thickness of the prongs

Facet structure and angles of the prongs

Angle of curvature of the prong and its angle (facing others) comparedto other prongs

How high the gemstone is set in comparison to the prongs and to theother gemstones relatively

Size and quality of any gemstone fallen or needs to be replaced

Placement of the gemstones in relation to the prongs

Are the gems overlapping each other or too near?

Are the prongs touching the gem or not secure?

Check size and shape of prongs next to it and around it and other areas.Are they same or for similar gems sizes and types?

Are the Prongs of same type? How many types? How many of each type?

The information mentioned above is exemplary and should not limit thescope of the invention. It should be clearly understood that thealgorithm can extract any other information from the first set of imagesrequired for verification of the jewelry item.

The frontal view of a jewelry item 400 is illustrated in FIG. 4. Thealgorithm calculates the dimensions of various parts from the image ofthe jewelry item 400. Exemplary dimensions include girdle and tablefacet of gemstone and placement (408), angle of prong when bent andcolor recognition and/or outline of tips and sides and height of prong(410), thickness of prong & angle of bending from side view (412),volume and point to point marking to check for changes (414) and facialrecognition of the ring using prongs, facet placement and inclusionsetc. as markers (416).

FIG. 6 illustrates a line diagram of a jewelry item 600 created by theprocessing unit 106 from the first set of images. Some of the exemplarydimensions calculated from the line diagram include girdle diameter(602), table diameter (604), pavilion depth (606), crown height (608),pavilion angle (610) and crown angle (612) of the gemstone, angle ofprong (614) and width of prong (616).

At step 216, the images, diagrams and extracted information is stored inthe storage module 104. At step 218, a second set of images are capturedat time interval T2 by the image capturing device 102. In a preferredembodiment of the present invention, the time interval T2 is the timewhen the jewelry item is returned from the jeweler after repairment. Thetime T2 can also be any arbitrary time at which the images of jewelryitem are captured and stored for reference, for example, to check forwear and tear in the jewelry item. Similar steps were followed forsecond set of images as for the first set of images to separate themetal part, gemstones and prongs and extract the information from imagesat step 220. The second set of images, diagrams and extractedinformation are also stored in the storage module 104.

In an alternative embodiment of the present invention, the different setof images of the jewelry item can be captured at regular intervals oftime to check for wear and tear and changes in the metal part andgemstones. These set of images can act as DNA map of the jewelry item toidentify the changes happened in due course of time. The image dataalong with a passkey is passed along to the next owner/handler of thejewelry item. Updates to the item or verification of the item (re-check)for any changes made between first photo and receipt of physicaldelivery can be carried out before acceptance of the item using newphotos and running the application to verify authenticity. If the oldphoto and new photo match and acceptance finalized, the passkey can beupdated with the new ownership. These images can be provided toinsurance companies, police, pawn shops and prospective buyers, etc.They can also get passkey from the user of jewelry item so that they canverify the authenticity, chain of ownership and accurate breakdown ofthe constituent products. The matching prongs and gemstone verificationcan allow the buyers to pay or value the metal part and gemstones muchcloser and more accurately. It can also let police and insurancecompanies to search the database if lost or stolen jewelry items arefound. It further gives pawn shops or money lenders a better idea toownership so that they have less risk while accepting to pawn the item.If a manufacturer makes multiple piece of the same jewelry item, itenables him to keep track of all the individual pieces accurately.

The first set and second set of images and the line diagrams createdfrom the images are superimposed at step 222 to verify a change in theimages between the time interval T1 and T2. FIGS. 7a and 7b illustratesthe line diagrams the jewelry item created from first and second set ofimages, respectively. FIG. 7c shows the superimposed image of FIGS. 7aand 7b to verify changes in the prongs settings of the jewelry item atstep 224. For example, the changes in distance between the prongs,height and thickness of the prongs, position of prongs with respect tothe gemstones, angle of curvature of the prongs and its angle comparedto other prongs, prong to prong height and seat size ratios, etc. Itshould be clearly understood that the above mentioned parameters areexemplary and the algorithm can take into account various otherparameters required for verifying the changes in prong settings. Thealgorithm calculates these changes on pixel-to-pixel basis from thesuperimposed image and displays all the changes on the output system110. The algorithm checks if the changes are within a predefinedthreshold limit at step 226 and generates an alert or a flag if any ofthe change is more than the threshold limit at step 228. Based on thechanges in the prongs, the algorithm calculates the changes in metalpart and gemstones of the jewelry item at step 230. For example, changesin dimensions of the gemstone (crown height, pavilion depth, tablediameter, pavilion angle, crown angle, etc.), size and volume of themetal part, etc. This enables the user to know what changes have beendone to the jewelry item during repair and whether these changes arewithin the acceptable limit. In an exemplary embodiment, if the gemstoneneeds to be replaced in the jewelry item, the user can verify from theoutput information if the replaced gemstone matches the value and sizeof the previous gemstone. If the size varies more than an algorithmicvalue, question can be raised to the jeweler who replaced the gemstone.If the algorithm includes the capability of automatic appraisal of thejewelry item, it evaluates the valuation of the jewelry item at timeinterval T2 and displays it to the user at step 232 on the output system110. This enables the user to compare the valuation of jewelry item attime T2 with the known valuation at time T1. The process stops at step234.

FIG. 8a illustrates a line diagram of the jewelry item with gemstoneremoved from the prongs at time interval T1. The software algorithmrunning on the processing unit 106 calculates the distance between theprongs 802, width of the prong 804 and angle of curvature of the prong806. The algorithm can also estimate the size and weight of a gemstonewhich can be best fitted between the prongs. The calculated values arestored in the storage module 104. In an exemplary embodiment, the imagesof jewelry item, the line diagram 800 and the calculated values are sentto a jeweler to know if the jeweler has the gemstone of required sizeand weight. If the required gemstone is available in stock, the jewelersends the image of the gemstone to the user for reference. The jewelryitem is then sent to a jeweler for setting the gemstone between theprongs. The jewelry item is received from the jeweler with fittedgemstone and its images are again captured. FIG. 8b illustrates a linediagram of the jewelry item at time interval T2 after a gemstone is setbetween the prongs. The software algorithm again calculates the distancebetween the prongs 812, width of the prong 814 and angle of curvature ofthe prong 816. The calculated values are stored in the storage module104. The algorithm calculates the difference between the correspondingstored values to verify the changes and displays it on the output system110. An alert or a flag is generated if the difference is more thanpredefined values (considering wear and tear during setting of thegemstone). In such a scenario, the user can question the jeweler toverify for the fitted gemstone and the loss of metal part.

FIGS. 9a and 9b illustrates the line diagrams of a jewelry item 900 attime interval T1 and T2, respectively. The prong size 904 has beenchanged at time interval T2 when the jewelry item is received from thejeweler after repairment. The software algorithm calculates thedifference between the prong sizes 902 and 904 and generates an alertaccordingly.

In an alternative embodiment, the user has the gemstone to be fitted inthe jewelry item. The user clicks an image of the jewelry item and thegemstone along with an object of standard size (for example coin). Thisenables the user to know the actual size of the jewelry item and thegemstone, irrespective of the image zoom size, by comparing it with thestandard size of the coin. FIG. 10 shows an image of a ring 1002, agemstone 1004 and a standard size coin 1006. The actual size of ring1002 and the gemstone 1004 can be easily known by comparing it with coin1006. The software algorithm running on the processing unit 106 alsoextracts the information of inclusions in the gemstone 1004 andidentification markings on the ring 1002 and gemstone 1004 (if any). Thealgorithm compares the information to verify the changes in the ring1002 and the gemstone 1004 between the time intervals T1 and T2.

In a further embodiment, the present invention provides a security inbuying a jewelry item through an e-commerce platform. The e-commerceplatform can provide 360° images of the jewelry items to a buyerincluding the gemstones, metal part and the prongs. The buyer can storethe images in the storage module 104 and process it using the softwarealgorithm running on the processing unit 106. The algorithm extracts allthe information from the images and stores in the storage module 104.After the actual receipt of the jewelry item by the buyer, the imagesare again captured and information is extracted by the algorithm. Theextracted information is compared with the stored information to verifyif the same jewelry item is received. This helps to detect any fraud bythe company of e-commerce platform or during the transit of the jewelryitem.

FIG. 11a illustrates the top view of a jewelry item with an initial setgemstone. FIG. 11b illustrates the top view of the jewelry item with thesame gemstone reset in the jewelry item. The FIGS. 11a and 11b showsvarious dimensions of the gemstone and the prongs. The tips of theprongs have also been changed before and after reset of the diamond asshown in FIGS. 11a and 11 b. It should be noted that the figures are forillustrative purpose and are not drawn to scale.

Diamond  size  (as  obtained  from  diamond  lab  report) − 6.36 − 6.39 × 4.13  mmRatio  of  the  diamond  size = 6.36/6.39 = 0.995305

Ratio of the lines passing through the diagonal of the gemstone:

E^(′)/12.70/10.00 = 1.27 F^(′)/12.80/10.10 = 1.267

Calculating the percentage difference between the calculated values anddiamond size

12.80 × 0.995305 = 12.734Percentage  (%)  error = (12.80 − 12.734)/12.80 × 100 = 0.549 < 1%

Therefore, the deviation in the prong setting will be acceptable if thedifference in corresponding values of FIGS. 11a and 11b is less than 1%.

Considering the prong to prong distance in FIG. 11a

$\begin{matrix}{{A\text{:}\mspace{14mu} 5.55 \times \left( {1.27,1.267} \right)} = {{{{Average}\left( {7.0485,7.03185} \right)} \pm {1\%}} = 6.969}} & 1. \\{{B\text{:}\mspace{14mu} 6.50 \times \left( {1.27,1.267} \right)} = {{{{Average}\left( {8.255,8.2355} \right)} \pm {1\%}} = 8.327}} & 2. \\{{C\text{:}\mspace{14mu} 6.00 \times \left( {1.27,1.267} \right)} = {{{{Average}\left( {7.62,7.602} \right)} \pm {1\%}} = 7.687}} & 3. \\{{D\text{:}\mspace{14mu} 6.50 \times \left( {1.27,1.267} \right)} = {{{{Average}\left( {8.255,8.2355} \right)} \pm {1\%}} = 8.327}} & 4.\end{matrix}$

Comparing the calculated values from the above equations 1, 2, 3, 4 withthe prong to prong distance in FIG. 11b

$\begin{matrix}{{A^{\prime}\text{:}\mspace{14mu}{\left( {6.80 - 6.969} \right)/6.80} \times 100} = {2.49\%\mspace{14mu}\left( {{not}\mspace{14mu}{acceptable}\mspace{14mu}{value}\mspace{14mu}{greater}\mspace{14mu}{than}\mspace{14mu} 1\%} \right)}} & 1. \\{{B^{\prime}\text{:}\mspace{14mu}{\left( {8.30 - 8.327} \right)/8.30} \times 100} = {0.333\%\mspace{14mu}\left( {{acceptable}\mspace{14mu}{value}\mspace{14mu}{less}\mspace{14mu}{than}\mspace{14mu} 1\%} \right)}} & 2. \\{{C^{\prime}\text{:}\mspace{14mu}{\left( {7.20 - 7.687} \right)/7.20} \times 100} = {6.765\%\mspace{14mu}\left( {{not}\mspace{14mu}{acceptable}\mspace{14mu}{value}\mspace{14mu}{greater}\mspace{14mu}{than}\mspace{14mu} 1\%} \right)}} & 3. \\{{D^{\prime}\text{:}\mspace{14mu}{\left( {8.40 - 8.327} \right)/8.40} \times 100} = {0.860\%\mspace{14mu}\left( {{acceptable}\mspace{14mu}{value}\mspace{14mu}{less}\mspace{14mu}{than}\mspace{14mu} 1\%} \right)}} & 4.\end{matrix}$

In general, 2 prongs are needed to be opened to remove the gemstone fromthe jewelry item. The percentage change in the prong to prong distanceafter resetting can conclude which prongs were opened during repairment.For the calculated values which have more than 1% deviation in the aboveequations, the software algorithm running on processing unit 106 willgenerate an alert or a flag which will be displayed on the output system110.

FIG. 3 illustrates a flowchart showing steps for online appraisal of thejewelry item. The process starts at step 302 and a jewelry item is takenat step 304. A set of images of the jewelry item are captured using theimage capturing device at step 306. The images are captured along withan object of standard dimensions, like a coin of standard size. Thisenables to calculate the exact size of the jewelry item and the angle ofpicture taken irrespective of the zoom view of the image. The set ofimages capture a complete view of the jewelry item. At step 308, thecaptured images are uploaded on an automated jewelry appraisal system.The automated jewelry appraisal system can be a software applicationrunning on a server computer, a client user computer, a mobile phone, acontrol system and a network router, switch or bridge. Alternatively,the software application can be running on a virtual computing system ofa cloud environment. At step 310, the user can provide manual inputs inthe automated jewelry appraisal system. For example, the user can inputphysical characteristics of the metal part and the gemstones likeweight, volume, dimensions, carat, cut grade, color rating, etc. Theuser can also upload jewelry certification and lab reports at step 312obtained from jewelry certification organizations like GemologicalInstitute of America, The International Gemological Institute, European,Gemological Laboratory, etc. At step 314, the software applicationautomatically detects the metal parts and the gemstones in the jewelryitem. The software application can use, for example, edge detectiontechnology or any other methodology to create a point-to-point diagramor a line-wire diagram of the jewelry item. At step 316, the softwareapplication extract more information from the images, like size, weight,carat, color grade, clarity, cut proportions, polish, etc. The automatedjewelry appraisal system access different online jewelry databases orwebsites, like zeales, Kitco, etc. to find jewelry items and gemstoneswith similar parameters, like color grade, prong styles, type of metal,design styles, etc. The data and valuation of these similar jewelryitems is received and compared with the user jewelry item at step 318.The software application then estimate the valuation of the jewelry itemat step 320 based on all the extracted information. At step 322, thegemstone or jewelry images and valuation is sent to an online commerceplatform for displaying and selling the jewelry item. The process stopsat step 324.

FIGS. 12a, 12b, 12c, 12d, 12e, 12f, 12g and 12h illustrates an exemplarymobile application of an online commerce platform for buying and sellingjewelry items. A buyer or a seller of the jewelry item can createhis/her profile in the mobile application and can login with thecredentials as shown in FIG. 12a . The platform provides variousfunctionalities to the buyer or seller like a ‘marketplace’ to buygemstones and jewelry items available from different sellers, “sell youritem” to sell its own gemstones and jewelry items, ‘scan’ to upload theimages and lab certificates of the gemstones and jewelry items, etc. asshown in FIG. 12b . FIG. 12c shows a marketplace of the online commerceplatform where the buyer or seller can view the details and pricing ofthe diamonds and jewelry items placed by different sellers on theplatform. FIG. 12d shows the functionality of selling diamonds andjewelry items by clicking and uploading its images. The seller can alsoscan and upload certificates and appraisal reports provided by jewelrycertification organizations like Gemological Institute of America, etc.The selling price of the item can also be inputted by the seller. FIG.12e shows a user interface for inputting the product details for placingthe product on the platform for selling. FIG. 12f shows a manufacturercorner for scanning and uploading certificates and appraisal reports bythe manufacturer of the product. FIG. 12g shows the functionality ofbuying a product from the online commerce platform. On selecting anavailable product, the buyer can see all the product details and can buythe product. The buyer or seller can also scan or input a product codeof the jewelry item or gemstone to find the product information such asprice comparisons and user reviews of similar products available onlineon the platform as shown in FIG. 12 h.

In accordance with these and other aspects of the disclosure, additionalconfigurations and variations of the disclosed systems and methods forverification of a jewelry item are further described herein.

The foregoing has included discussion about a diamond's girdle, like itsbearding, hills, valleys, small dents, etc.; however, such features arenot used for performing gemstone verification.

Gemstone verification can involve gemstone characterization whichincludes operations for analyzing gemstone images and other measureddata to detect and extract (e.g., measure or derive) a uniquecombination of gemstone features that are usable to identify anddifferentiate each unique gemstone. Gemstone characterization can alsoinclude generating a record of gemstone identification information,which can include one or more digital visual representations of gemstonefeatures and/or other encoded information representing gemstonefeatures. Such a record can be stored in a database record and is usableto perform further security verifications and operations. Gemstoneverification can also include authenticity verification, which involvesmatching all or part of a query set of images and/or gemstoneidentification information against one or more previously registeredrecords for the purposes of identifying or authenticating a gemstone andto determine whether any unauthorized alterations have been made to agemstone. Gemstone verification can also include identifying any changesmade to a gemstone over time and recording information concerning thesechanges in the database record for the gemstone to provide a completeand verified history of the gemstone and changes over time. Inaccordance with one or more embodiments, the system is furtherconfigured to utilize the unique combination of physical characteristicsof a gemstone including any combination of the foregoing as a securityfactor to perform or authorize the gemstone wearer/owner to performvarious secure operations and transactions.

Although the exemplary embodiments of the system 100 described hereinare primarily discussed as performing gemstone verification based onphysical characteristics of the girdle of a gemstone, the system cansimilarly perform verification based on extracted physicalcharacteristics of other parts of the gemstone (e.g., its facetstructure, inclusions and the like) as well as other parts of a jewelryitem that the gemstone might be combined with including a metal part,gem holders that hold the gemstone to the metal part, such as prongs.Characterization of the girdle and its use in uniquely identifying agemstone is a salient aspect of several embodiments in accordance withthe present disclosure.

In accordance with one or more embodiments, the system 100 is configuredto image, analyze and measure physical characteristics of a gemstone,particularly its girdle, and utilize the measured features, differencesin the gem girdle and minute detailed differences that occur along thelength of the girdle to identify a gemstone. In this regard, the system100 can be configured to use the image capturing device 102 to captureone or more images of a gemstone including its girdle, which typicallyextends peripherally about the side of the gemstone. Image(s) generallyrefers to a digital visual representation of the gemstone that can becaptured using photo, video, laser-based tomography and other suchthree-dimensional scanning devices. Images can be captured under normallighting and/or under special lighting conditions. For example, FIG. 13Ais a photograph captured of part of a gem's girdle captured using adigital camera from a top-side perspective view (left image) and FIG.13B is a photograph of the same gem's girdle from a bottom-sideperspective view (right). The simplified gemstone images of FIGS. 13Aand 13B, illustrate some exemplary features pertaining to the girdle1305 of the gemstone including, the top and bottom edges 1350 where thegirdle intersects with adjacent facets 1330, girdle bearding features1310 that are typically near the edges of the girdle (e.g., either onthe face of the girdle or adjacent facets 1330), natural or man-madeimperfections 1320.

The images can be stored in a storage module 104 and provided to aprocessing unit 106, which, by executing one or more of the exemplaryprocessing methods further described herein, is configured to extractinformation from the gemstone image, particularly, informationconcerning features (i.e., physical characteristics) of the girdle. Theprocessing unit can also extract features of other parts of the gemstonein addition to the girdle, say, neighboring facets. As noted,optionally, additional features of the gemstone or jewelry item as awhole can be measured as well. Physical characteristics of a gemstone orjewelry item extracted from the images can also be generally referred toas herein feature data.

More specifically, the processing unit 106 is configured to extractunique set of characteristics of the gemstone using the girdle andfacets of the gemstone that define the edges of the girdle.Additionally, the processing unit can be configured to create a line toline and point to point diagram or 3D image representation of a lengthof the girdle. FIG. 13C is an exemplary line to line and point to pointdiagram of the girdle shown in FIG. 13A wherein the line to line andpoint to point diagram is overlaid over the image of FIG. 13A. FIG. 13Dis an exemplary line to line and point to point diagram of the girdleshown in FIG. 13B wherein the line to line and point to point diagram isoverlaid over the image of FIG. 13B. In the exemplary diagramsillustrated in FIGS. 13C and 13D, girdle bearding features are shown aslines, points or cross-hatched areas 1315, girdle features such asnatural or man-made imperfections are shown as solid lines or points1325, edges of the facets are shown as solid lines 1335, and edges ofthe girdle are shown as solid lines 1355.

From the diagram, particularly, the lines that mark the top and bottomedges of the girdle, the processing unit 106 can be further configuredto identify the valleys and peaks of the girdle and measure the distancebetween those features, e.g., the height of the girdle at one or morepoints. The processing unit can also be configured to extract (e.g.,measure or quantify) other physical characteristics of the girdle basedon the images and/or resulting diagram including, for example, thechange in the girdle height along the length of the girdle, thewave-like pattern defined by one or more the girdle edges, theparticular location, size and pattern of junctures between adjacentsegments (e.g., facets) of the girdle. Additionally, the processor canbe configured to measure girdle bearding features. In an embodiment, theprocessing unit can implement a trained Neural Network, which can betrained on training images to detect and classify features of the girdleincluding bearding, scratches, inclusions, and the like. Other imageanalysis and feature detection algorithms can be used to identify girdlefeatures like bearding. The processing unit can be configured to measurevarious characteristics of individual features and their arrangement.For example, the processing unit can measure the length and shape anddepth of a bearding feature or inclusion based on pixels. The processingunit can be configured to measure one or more features based on relativedistances between features and position, extrapolate the shape of thefeature and match with another. Features such as bearding can also bemeasured in actual size, shape and location values. Gemstone sizemeasurements can be measured by the processor from the imagery ormeasured using other measurement devices and provided as an input to theprocessing unit. Exemplary characteristics of one or more of the girdlefeatures that can be measured include, location/coordinates, length,width, shape, color, opacity, type of feature (e.g., bearding), relativeposition to other features near it, or on an opposite side of the gem toit, if any.

Furthermore, the system 100 can be configured to utilize these and othergirdle characteristics as a unique identifier for a gemstone on the onehand and as a security factor on the other. Similarly, the uniquecombination of girdle features can be used to differentiate between anduniquely identify gems, and to evaluate whether there have been changesto a particular gemstone's features over time.

In addition to capturing the profile of a girdle, natural markings maypreexist on a gem and cutting of a gem can also leave man-made ormachine made etches in a gem. These features are sometimes only visibleunder certain lighting conditions, yet these markings define a uniqueidentification fingerprint that is almost impossible to replicate.Accordingly, the processing unit 106 can be configured to record suchfeatures and use them for identification of a gem to a very high degreeof accuracy and security. In particular, special lighting conditions canbe generated by the system 100 using various types of light emittersduring the imaging step. Special lighting can include, for example, asimple white light or laser light that is shined at a specific angleonto the gem and that causes these manmade or natural features tostand-out in the girdle images. Alternatively, special lighting cancomprise a light pattern or a filtered light source that has a range ofwavelengths filtered out or band-passed so as to differ fromconventional white light. Additionally, filters can be provided betweenthe light emitter and gemstone or between the gemstone and camera so asto reduce excess glare and screen out blinding light from the camera.Identification of a gemstone based on unique natural and man-mademarking features on the girdle can avoid the need for costly equipmentpreviously utilized for identifying a gem using laser inscriptions,which can be easily removed or erased.

The system 100 can be configured to implement the exemplary methods forcapturing gemstone features, analysis and characterization using a basicdigital image camera such as a smartphone camera. However, more detailedinformation can be revealed by using a high-definition camera, such asone that has a macro or high-powered zoom. As noted previously, theidentifier markings identified through analysis of the gemstone image,among other information extracted or generated from the raw or processedimages and diagrams, can be stored in a database, for instance, externalstorage 112 and/or the storage module 104 shown in FIG. 1.

Preferably the system is configured to obtain images includingidentifier markings for the entire girdle. As used in this disclosure,the system is or can be configured, in each case, by providing code froma memory which is executed in a processor to perform the functionsdescribed herein. Thus, when the system is configured to obtain imagesincluding identifier markings for the entire girdle, the processor isconfigured by code for that purpose.

Identifier markings can also be obtained for all facets of the gem.Additionally, the system can be configured to break up the images intosmaller images. For instance images can be broken up based on variousfeatures including, for example, at each juncture of the girdlesegments, the meeting of facets to the girdle or meeting of facet tofacet among other features like inclusions or open surfaces in a gem.The processor can execute code to identify the junctures and divide theimages in memory on that basis for analysis or for storage. The raw andprocessed images, and the generated visual representations of thegemstone, can be stored as a whole 360 degree point to point and line toline diagram. However, even a small juncture or part of a previouslystored image of a girdle and/or faceting can be utilized for gemstoneidentification. For instance, a small juncture of a known/storedgemstone image can be checked against a current query image and, if amatch is found, the gem is identified. The system can also be configuredto identify a match between gemstones by overlapping girdle images or“marker images,” which refers to the point to point and/or line to linediagrams representing the unique combination of features used todifferentiate one gem girdle from another.

Further, in accordance with one or more of the embodiments discussedabove, if the gem is set in a setting (i.e., “pronged”), an image of theprong and the placement of the prongs and the girdle can also becaptured. The system can be configured to use such additionalinformation as an even more specific test to determine whether a gem hasmoved, been replaced or otherwise altered from its original settingusing point to point and line to line comparison and the prongs as apoint of identification.

In accordance with one or more embodiments, the system 100 can beconfigured to capture one image or 360 degree video of the gemstoneunder lighting by a light emitter 180 (as shown in FIG. 15) on andanother with it off. The system 100, particularly the processing unit106, can be further configured to perform a comparison between thelighting on image and lighting off image to further identify any hiddengirdle features that present as differences between lit and unlit girdleimage. The images can be compared in either parts or the whole, asneeded. Additionally, during gemstone identification, a particularportion of a reference image or diagram can be located in a database andcompared to the corresponding portion of the query image or line to lineor point to point diagram. For example and without limitation, imagecomparison can be performed using pixel comparison methodologies inwhich an array of pixels is used for the actual comparison of the queryimage and the reference image it is being compared to. In addition oralternatively, point to point and line to line diagrams can be generatedfor respective image segments and compared.

By way of further example, a Neural Network can be trained on the beforeimages of a gemstone to extrapolate pertinent feature data and save itto the database record. The new images of a gemstone are similarlyanalyzed to gather the feature data and then perform an image comparisonagainst the previously obtained/trained dataset. Matching can beperformed incrementally, for instance, if a match of a point-of-interestis found, the processor can be configured to compare more such pointsfrom the images.

In one or more embodiments, the processing unit can be configured toimplement image comparison as a function of which angle the query imageshave been taken from and perform matching against recorded imagescaptured from similar angles.

Additionally, the gemstone image comparison can be performed using pointto point (P2P) and line to line (L2L) and diagram to diagram (D2D)comparison methods. The P2P L2L and D2D can be used individually ortogether or in various combinations, depending on what information is ofinterest and from what angle/s the images were taken.

The collection of feature data points that are usable as a gemstoneidentifier can be stored in the database record in the form of a diagramthat can be physically reproduced for a user to view. In addition, oralternatively, the feature data set can be encoded in the Neural Networkor computer program (e.g., in a numerical form, as a mathematicalformula or other such machine readable encoded representation). Theprocessing unit can be configured to generate the identifier byexecuting the steps of: finding and segmenting parts of the jewelry itemand tagging them. Then the processor can identify points of interest(POI), key POIs being the gem, girdle, inclusions, facet lines, beardingon girdle or other dents and nicks, culet center and facets leading toit, inscription on gem etc.

As noted, additional identifying and unique features can be found inaccordance with one or more of the embodiments. For the jewelry item,for example, the feature data can include the angle of prongs, alocation where the prongs touch the gem and the girdle touches theprongs, color of the metal, scratches and other unique features, thestamping on the metal and the like.

In one or more embodiments, the processing unit can be configured togenerate the P2P and L2L diagrams in various ways. For instance, if twoPOI points are found, a descriptor line can be drawn connecting them inmultiple ways including: 1) a straight line; 2) a line going through anobject within the image; 3) an inverted “V” shaped line or line of someother shape; 4) an imaginary line or real line linked to that descriptorline and having a fixed number of pixels or distance from thatconnection. Subsequently, such information concerning the POIs can beused to train the matching algorithm to use the same POIs and distancesfor triangulation and processing of subsequent images. For instance,when a new set of one or more gemstone images (e.g., video, sonar,laser, any sort of image type) for matching against a database record,the processing unit can be configured to identify the same POIs and,using the same pixel distance or line lengths, determine how and wherethe same points match between both image sets. Since the query image andreference image might not be taken at the same angle and distancerelative to the gemstone, the exemplary line to line and point to pointand diagram of the POIs (e.g., edges) can provide enough identifierpoints for the processing unit to match, reject or determine a partialmatch to a prescribed degree of confidence.

In one or more embodiments, the processing unit can be configured tofind POIs on various elements of the jewelry item, align those POIs in aprescribed way by rotating scaling or otherwise adjusting the imagery(and if necessary capture additional images at one or more positions orangles), and measure the POI to POI lines/distances, sizes etc.

As previously noted, in one or more embodiments, the processing unit canbe configured to analyze a gemstone image using a reference point orobject, such as a object in the foreground or background of the image(e.g., a coin in the image used as a size reference, a circle having aknown size provided on the platform of the gemstone mount, a marking ona gemstone holder or imaging station) to obtain bearings, performaccurate alignment, scaling and size measurements.

In accordance with one or more embodiments, other features of a diamondcan be imaged and analyzed and used to generate a point to point andline to line diagram at various stages in the manufacturing process. Theparticular combination of parts of the gemstone that are imaged andanalyzed throughout the lifecycle of a gemstone can vary depending onwhich stage a gemstone is at between discovery at the mine through theretail and ownership stages. For example, FIG. 14A is an image of arough diamond captured after being split into two portions. FIG. 14Bincludes a point to point and line to line diagram of detected featuresof the outer-surface and the edges of the two portions overlaid on theimage of FIG. 14A. As shown in the exemplary diagram of FIG. 14B, edgesof the diamond are shown as solid bold lines 1435, and dashed lines anddash-dot-dash lines identify the correspondence of certain surfaceedges/points 1440 on one half of the rough diamond to surfaceedges/points 1440 on the other half of the rough diamond aftersplitting.

Additional details concerning exemplary methodologies used by the system100 for imaging the girdle, identifying unique features, using thosefeatures to identify and verify a jewelry item are further describedherein.

As noted, the system 100 can be configured to image and analyze featuresof a gemstone including girdle features using a suitable image capturedevice 102 having, for example, video, image, and/or laser-based imagingtechnologies. For instance, the image capturing device can include adigital video camera that has high-resolution imaging capabilities. If,however, a low-resolution image capture device is used, the system,particularly the processing unit 106, can implement image processingalgorithms to analyze the image, isolate features and otherwise updatethe images to more suitable resolution for further analysis.

In one or more embodiments, the image capture device can also beconfigured to capture images under moving or vibration conditions, suchas those conditions that can occur while a gemstone is being cut with acutting wheel. In such a configuration, the image capture device ispreferably able to auto focus to improve imaging quality while thegemstone is moving or vibrating. In one or more configurations of thesystem, one or more such image capture devices can be used, depending onthe type of process being implemented. For instance, differentconfigurations of the system 100 can be provided at the mine stage(e.g., for imaging rough diamonds) or manufacturing stage (e.g., forimaging and analyzing gems during one or more of the cutting, cleaving,polishing, bruiting, rounding and/or polishing the girdle stages etc.).As such, the imaging device needed may vary based on the process beingperformed at a given stage. However, preferably the imaging devicecaptures imagery that is very focused and high resolution and in view ofany required lighting features.

In one or more embodiments, the system 100 can be configured tocontrollably move the gemstone during imaging, for instance, using acomputer-controlled gemstone support table. It can be preferable toimage the girdle while it is held at a tilted angle relative to theimage capturing device, or while continuously tilting the girdle througha range of angles between a top perspective and bottom perspective angleso that the light bounces off the girdle and minute details of thegirdle features including inclusions, cutting patterns, girdle facets,bearding etc. are more visible. Similarly, the system can be configuredto make these and other features such as facet lines and natural,cutting or wear and tear created differences and identifier points morevisible and brought forth through controlling lighting angle and/ortilting motion and imaging angle.

For example, exemplary steps for imaging the gemstone while rotatingand/or tilting the gemstone performed using the system 100 can include:positioning the gemstone at a starting position in which the opticalaxis is pointed directly at the girdle and perpendicular to thegemstone's vertical axis, which extends from the bottom of the gemstoneto the top through the center of the gemstone; controllably rotating thegemstone 360 degrees about its vertical axis between a start/stoplocation; and capturing imagery continuously during rotation or at aprescribed number of rotational positions. Additionally, at one or moreof a plurality of rotational positions, the system can be configured tocapture one or more images while a) controllably tilting the gemstonesuch that the vertical axis is at least partially angled away from theimaging device (e.g., providing a bottom perspective view of the girdleat a given rotational position); b) controllably tilting the gemstonesuch that is vertical axis is angled at least partially toward theimaging device (e.g., providing a top perspective view of the girdle).Images can also be captured for one or more angles in the range ofangles between the tilted-away and tilted toward positions. FIG. 15 is ablock diagram of the exemplary system 100 shown in FIG. 1 furtherillustrating a light emitter 180 and rotating/tilting gemstone mount 185that are in communication with the processing unit 106 and controllableby the processing unit in accordance with one or more embodiments.

While it can be preferable to automatically control rotation, tilt,imaging angle, and lighting angle automatically by adjusting theposition of one or more of the camera, the lighting device and thegemstone mount, more basic configurations of the system 100 can beconfigured to perform the imaging, girdle-based analysis andidentification operations without such computer-controlled enhancements.

As noted, images of a gemstone can be captured at various stages duringthe mining, manufacturing, marketing/retail and ownership stages. In oneor more embodiments, changes made to a girdle during manufacturing (orother gemstone and jewelry item part for that matter), can be capturedin a before and after image. Additionally, a before and after imageprocessing method can be implemented by the processing unit 106 whichincludes at least one processor and a memory to store code to configurethe processor, and, hence, the system described herein, to create a new“spliced image” that includes the changed portion(s) of a gemstone part(e.g., girdle, facet, etc.). The “spliced image” can be created usingconventional photo or video editing and splicing and gluing operations.The “spliced image” can be created in a virtual way such as building a3D imaging model of the gemstone part from the before image and thenslicing and gluing to that model, a virtual representation of the partthat is generated from the after image showing the modified part.Additionally, the processing unit can be configured to implement acombination of these processes to show an image of the changed part onthe virtual image or with the virtual spliced part on a real image etc.

In one or more embodiments, the image capturing device 102 can be placedin the gemstone processing station (e.g., the cutting or gemstone mountstation) or mounted externally, depending on the requirements of theprocess performed at a given processing station. The system 100 can beconfigured to automatically start and stop the imaging process inresponse to a pressure-release, manual, automatic, or some other methodof triggering. The image capturing device 102 provides the images to theprocessing unit 106 and/or stores the images in a database of the system100 at which point the processing unit 106 can perform the variousprocesses to capture details of the gem (e.g., the weight or where it'sfrom) and identify physical features.

As noted above, the system 100, particularly the processing unit 106, isconfigured to identify and measure various features of the gemstonegirdle. These features can include, girdle bearding, the wave-like lineformed by the top and bottom edges of the girdle, inclusions and othermarkings. The system is also configured to measure and identify thegirdle itself, the lines corresponding to the top and bottom edges ofthe girdle and the wave-like pattern of peaks and valleys the edgesdefine, and the distance between the two edges. The system is alsoconfigured to measure the dimensions and arrangement of girdle segments(e.g., facets), the spacing between girdle segments and the size,spacing and pattern that the junctures (e.g., spaced-apart verticaledges between segments) occur along the girdle length. Detecting edgestop/bottom and junctures can be performed using an edge detection imageprocessing methodology. Further the system is configured to identify andmeasure the small features or and differences created by the cuttingprocess or naturally. Further the system 100 is configured to create thepoint to point and line to line diagrams representing these and othergirdle features.

The processing unit 106 can be configured to generate a L2L or P2Pdiagram using a neural network or algorithm programmed to draw linesjoining identified points of interest. A line can be a straight line ora bent line, circles or concentric circle lines, or any other prescribedline shape. Lines and points can be represented visually in a diagram.Lines and points can similarly be mathematically represented (e.g., as aformula, values for location and length and direction, a vector, etc.).

The diagram generated for a gemstone is preferably a combination L2L andP2P diagram in which gemstone features are represented by a combinationof lines and points. In some instances, a line (e.g., an edge of thegirdle or facet or an elongate feature) might not be identifiable orfully identifiable in an image or might be cut off in the image or dueto the angle at which the image was captured. Accordingly, points ofinterest are identified and used in conjunction with the lines toprovide a L2L and P2P diagram. However, it should be understood thatdiagrams of points and lines can be separately provided. The linediagram also can also be used to help isolate whole or parts of thejewelry item or gem.

The processing unit can be configured to generate the L2L and P2Pdiagram by detecting the various features of a gemstone and representingdetected edges of a gemstone and other elongate features as lines andrepresenting smaller point-like features as points. It should beunderstood that the ends of lines can similarly define points in thediagram. As noted, the L2L and P2P diagram can also include additionallines which the processing unit draws between lines orpoints-of-interest.

In some embodiments, the processing unit can be configured to generate adiagram of a girdle by “drawing” one or more lines along the girdle orparallel to the girdle. In one example, three lines can be drawn, onealong the top edge, one along the bottom edge and one in the middle ofthe girdle. Additionally, the processor can be configured to draw linemarkers at the location of inclusions, natural or manmade markings onthe girdle, girdle bearding and other such features.

In accordance with one or more embodiments, the system 100 is configuredto use lighting and tilting and other means to capture more informationabout the physical features of a gemstone than just taking a video orphoto. Moreover, the system can comprise various combinations ofdevices, e.g., tilting, lighting and imaging devices that can betailored to the type of process being done.

In accordance with one or more embodiments, the exemplary methods foranalyzing a gemstone include specific processes for processing theimages. These image processing operations can make it easier to match oridentify a gemstone even if the image is only a small section of thewhole gem like the girdle or even a smaller section (e.g., small sectionor sections of a girdle). Preferably, one or more visible sections of agirdle may need to be used to increase accuracy, especially if a gem isset in a holder of sorts during the image capture process.

In accordance with one or more embodiments, the processing unit 106, canbe configured to implement methods for analyzing a gemstone that includeoperations for breaking a gemstone down into smaller identifiable parts,and each such part can be searched for in the database.

Additionally, in the situation where the gemstone is known and analysisis being performed for the sake of change monitoring, this can cause theprocessing unit 106 to splice in the information relevant to the newchange.

As noted, the processing unit 106 can be configured to generate adiagram of a whole jewelry item, respective parts of a jewelry item(e.g., the gemstone and opposed to the metal part) and/orsub-parts/segments (e.g., a girdle or girdle segment diagram).Accordingly, the processing unit can be configured to segment an imageof a jewelry item comprising a gemstone and a metal part (e.g., a ringand prongs that the gemstone is set in) into parts. For instance, theprocessing unit 106 is configured to segment the image into a gemstone,and metal part, and then segment the metal part into prongs and shankand inclusions and metal scratches etc. Similarly, the portion of theimage(s) classified as the gemstone and segmented, can be analyzed tofurther classify and segment the gemstone image into sub-parts/segmentssuch as the girdle, girdle segments and other gem parts. The processingdevice thus is configured to, using one or more classifiers andsegmentation algorithms, classify the various parts of a gem from theimages, separate out the respective parts in the images. The classifiersand segmentation algorithms can assist the training of the neuralnetwork used to identify distinguishing features, characterize agemstone (or other part of the jewelry item) and assist the matchingoperations performed by the neural network.

In accordance with one or more embodiments, if the full image of acomplete gemstone is not available, the processor is configured toperform analysis and matching using a partial image. In particular, theprocessing unit executing a classification algorithm, can be configuredto identify a particular part of the gemstone depicted in the image,say, the girdle or a ¼ girdle segment. Accordingly, the feature data forthe girdle can be extracted in accordance with the exemplary methodsdescribed herein. Additionally, in accordance with the exemplary methodsdescribed herein, the feature data from the query image can be comparedmatched against one or more references in the database, wherein thefeature data is only matched against the corresponding girdle featuredata for the respective references.

As can be appreciated, there are various technical challenges toovercome in order to image and analyze a gemstone girdle. For example,it is not easy to find the small differences in the facets withoutlighting and tilting the gem or the use of a high-powered camera device.Also focusing on the girdle point and finding and splicing and matchingor replacing captured images exactly where any changes have occurred orwhere any part of the gem is hidden.

Additionally, because finding size and distance or depth perception froma 2D image can be challenging, the system 100 can be configured togenerate a 3D model from images captured at multiple angles. In additionor alternatively, the system can be configured to gauge depth perceptionbased on a reference to a known object, like a disc or something thatreflects off the metal or gem thereby allowing calculations of depth andangles. Accordingly, the system can be configured to automaticallyidentify and analyze the known object's reflections or shadows or degreeof refraction and determines the needed information relating to depthand angle.

In accordance with one or more embodiments, the system 100 can havevarious configurations depending on the application and requisitedetail. In one configuration the system can utilize conventional phonecameras and optical magnifying lenses and other instruments like a gemholder or tweezer to tilt and take images. In addition or alternatively,additional light devices and laser pointers could be used as well.Alternatively a box-like mechanical gemstone mount or instrument couldbe utilized. As noted above, in a more complex arrangement the systemcan comprise a dedicated imaging station including one or morehigh-resolution imaging devices, a computer-controlled gem holder,lighting equipment and the like.

To recap, the exemplary systems and methods for verification of agemstone include steps for mapping out the features of a girdle, aloneor in combination with other gemstone parts.

As can be appreciated, the exemplary system 100 identifiesdistinguishing features of a gem (and optionally the jewelry item it isset in) and encodes those distinguishing features in one or more digitalrepresentations of the gemstone that can be stored in a database recordfor the gemstone. For instance, the digital representation of the gemcan be a diagram representing the distinguishing features in the form ofpoint to point and line to line diagram. Additionally, the image(s) canbe stored in raw and/or processed form in the database record.Furthermore, information input into the system or derived from theimagery and analysis of the gemstone can be stored to the databaserecord to facilitate subsequent searching, matching and other suchoperations. That additional information can include, for example, otherphysical characteristics of the gem or jewelry item, the owner of thegem and the like. By way of further example, the additional informationcan also include codes representing the characteristics ofdistinguishing features of the gem.

In accordance with one or more embodiments, the system 100 can beconfigured to utilize virtual reality or augmented reality photo/videodevices to perform the imaging, analysis and display of information to awearer or user/controller of the augmented reality device. The imageprocessing and analysis can be performed inside the AR machine itself,using a separate computing device in communication over wifi or Ethernetlike a mobile phone. In addition or alternatively, images can be sent toanother computing device to perform the image processing and analysis.More specifically, the system can be configured to use an augmentedreality device having a camera to capture real time images of jewelryitems and make real time calculations and predictions. The augmentationallows the user to move around, either the jewelry item or the imagetaking device. Based on the real-time analysis of the images, the system100 can be configured to guide the user, via the AR device display oraudio output, where next to move the device or the item and verifywhether the movement enabled the system to detect the necessary data.For instance, the user can be instructed to move the imaging device overa gemstone set in a gold set ring being held by 4 prongs (e.g., usingaudio or visual instructions to capture a top view then side view etc.).As the system receives and identifies the information from the imagesnecessary to identify the item, like the gem top, side, the girdle,prongs, and any small inclusions, scratches or the physical tag that mayhave a bar code or some sort of identifier, the system can in real timepredictively identify the item (e.g., associate it with an existinginventory stock item/number) and predict real-time pricing and otherdetails. The information gathered or generated by the system, such asthe line to line and point to point diagrams, prong setting, girdle andother features used to identify or valuate the item can be output to theuser, as well as pricing and identification information. This way, thesystem including the augmented reality input/output interface providedcan be used to provide the user with a tag-less ID of the item andvirtual price tag. While the foregoing embodiment of the system isconfigured to work with a dedicated AR device such as AR glasses withimaging and display functionality, a smartphone, tablet or other suchmobile computing device can similarly be used with the system to providean input and output interface with AR functionality.

In accordance with one or more embodiments, the system can be configuredto perform imaging process multiple times and at varying angles forbetter results. Features such as gemstone weights, colors, place, timeof first instance, can be manually or automatically determined by thesystem using connected devices such as scales or automatic colordetection systems. Then a point to point and line to line diagram and 3Dmodels can be created to map the features.

In accordance with one or more embodiments, the system can be configuredto rotate the gemstone during imaging based on the faceting.Furthermore, each facet can be separately illuminated and the facetjunctions can be used as markers to find the junctions and rotateaccordingly. The system can be configured to adjust the light tointersect each respective section by increasing or decreased lightwidths so as to match the size of the respective junction.

In accordance with these and other aspects of the disclosure, additionalconfigurations and variations of the disclosed systems and methods arefurther described herein. According to a further embodiment, featuresand functions of the systems and methods for performing gemstoneanalysis and verification are adaptable to provide a tool for processingof multiple loose gemstones using enhanced computer vision techniques(i.e., image-based analysis techniques).

FIG. 16 illustrates an exemplary system 1600 for processing multipleloose gemstones using image-based analysis techniques. The system 1600comprises the same basic components as system 100 including, an imagecapturing device 102, a memory/storage module 104, a processing unit106, input system 108 and output system 110. Whereas the system 100 hasbeen shown and described as being configured to image and analyze one orperhaps a few gemstones at a time, the exemplary system is configuredfor performing image-based analysis of relatively large quantities ofgemstones, say, for example and without limitation, on the order of tensor hundreds of gemstones.

By way of overview and introduction, as further described herein, theexemplary system 1600 for processing multiple loose gemstones can beconfigured to capture one or more images of a relatively large quantityof loose gemstones (e.g., 150 gemstones). As shown in FIG. 16, thegemstones 1650 are provided on a flat surface 1685 of a workspace (the“worksurface” 1685) where the gemstones are being processed. The systemis further configured to analyze the one or more images in order tofacilitate one or more of a variety of manual and/or automated gemstoneprocessing operations. The gemstone processing operations can include,for example and without limitation, counting the gemstones, separatingout specific quantities of gemstones, classifying the gemstones based onone or more of a variety of physical characteristics, sorting thegemstones, selecting gemstones having specific characteristics, and thelike. Although the gemstones are placed on a worksurface, such as aplatform, it should be understood that the gemstones can similarly beimaged, analyzed and processed while being held by a human hand or othersuitable gem holding device.

FIG. 17 is a flow chart of an example method 1700 for processingmultiple loose gemstones according to an embodiment of the presentdisclosure. The method 1700 and other methods disclosed herein can beimplemented using the system 1600 of FIG. 16. However, it should beunderstood that portions of this and other methods disclosed herein canbe performed on or using a suitable custom or preprogrammed logicdevice, circuit, or processor, such as a programmable logic circuit(PLC), computer, software, or other circuit (e.g., ASIC, FPGA)configured by code or logic to carry out their assigned task. Thedevice, circuit, or processor can be, for example, a dedicated or sharedhardware device (such as a laptop, a workstation, a tablet, asmartphone, part of a server, or a dedicated hardware circuit, as in anFPGA or ASIC, or the like), or computer server, or a portion of a serveror computer system. The device, circuit, or processor can include anon-transitory computer readable medium (CRM, such as read-only memory(ROM), flash drive, or disk drive) storing instructions that, whenexecuted on one or more processors, cause portions of the method 1700(or other disclosed methods) to be carried out. It should be noted thatin other embodiments, the order of the operations can be varied, andthat some of the operations can be omitted. The electronics can alsoinclude a user interface equipped with a touch screen (e.g., a touchscreen of the image capturing and gem evaluation device, such as a touchscreen of a mobile phone) to permit computer interaction.

In the example method 1700, processing begins with capturing 1705 one ormore images of the gemstones using an image capturing device. In someembodiments, this can be performed using the image capturing component102 taking a top-plan view photo of the worksurface 1685 having theplurality of gemstones 1650 resting thereon. In some embodiments, thecapturing includes capturing two or more images of the gemstones. Thetwo or more images can be taken from one or multiple angles, such asfrom directly above the worksurface, from a top-side perspective of theworksurface, and/or from any other suitable vantage point. The digitaldata in the two or more images are stitchable together for analysisaccording to the techniques described herein. In another example, datais extracted and analyzed from the images separately, or from athree-dimensional composite image created from the separate images.

At step 1710, a first image processing step is performed by theprocessing unit 106 on the one or more images. In some embodiments, step1710 can include performing edge detection on the one or more images.The edge detection algorithm detects edges of the gemstones in the imageand possibly edges of other objects depicted in the image. Step 1710 canfurther include generating and storing information concerning thedetected edges such as a diagram representation of the detected edges,for example, a point-to-point and line-to-line diagram. In someembodiments, step 1710 can include image pre-processing steps thatfacilitate operation of subsequent gemstone detection steps.

At step 1715, a second image processing step is performed by theprocessing unit 106 to detect individual gemstones within the one ormore images. In some embodiments, the second image processing step caninclude processing the one or more images using a neural network trainedto detect gemstones within an image. In particular, the neural networkcan be trained to detect objects within an image that are gemstones,classify each as a gemstone. Step 1715 can also include recordinginformation for each detected gemstone, such as a reference number oridentifier (e.g., gemstone number X of total gemstone count Y) itslocation within the image and/or corresponding location on theworksurface 1685 and other characteristics determined from theimage-based analysis. In some embodiments, the neural network can alsobe configured to identify and classify other types of non-gemstoneobjects, if any are depicted in the image.

In some embodiments, the neural network can be trained to analyze theraw image. Additionally, in some embodiments, the neural network'sprocessing of the image can be further informed by the result of step1710 such as the diagram of detected edges. For example, the image canbe enhanced by the edge diagram and the composite image and diagramprocessed by the neural network. By way of further example, the detectededge information can be provided as an input to an attention mechanismof the neural network such that gemstone detection and classification isperformed by the neural network on the image as a function of the resultof the edge detection step 1710.

In some embodiments, at step 1720, the processing unit can performadditional image-based analysis operations on the one or more images orrelated diagrams to determine or measure physical characteristics of theplurality of gemstones and respective gemstones. The result of suchoperations can facilitate the classification or differentiation ofgemstones and can further enable the processing unit 106 to categorizegemstones based on physical characteristics determined from theimage(s).

At step 1725, the processing unit performs one or more of a variety ofgemstone processing operations as a function of the detection andclassification of the gemstones. The gemstone processing operations canbe fully automated, partially automated, manually performed based oninformation output by the system via the output interface 110 or acombination of the foregoing.

For example, in an embodiment, at step 1725, the processing unit can beconfigured to count the number of gemstones identified within the image.In addition, or alternatively, the processing unit can be configured tofacilitate further manual and/or automated gemstone processingoperations, including, for example and without limitation, separatingout specific quantities of gemstones, sorting the gemstones and othersuch operations. Accordingly, the processing unit 106 can be configuredto facilitate processing operations at step 1725 based on parametersinput by an operator including, for example, the particular operation toperform, and criteria for the operation.

As one practical example, the processing unit 106 can be configured toreceive an input from the operator that one hundred gemstones are to beseparated out from the assortment of gemstones 1650 on the worksurface1685. Based on the total number of gemstones counted, the processor canbe further configured to identify a sub-set of gemstones that meet theinput parameters (e.g., one hundred gemstones).

Additionally, the processing unit 106 can be configured to provide anaugmented-reality processing tool by outputting, on a visual display ofthe output system 110, a real-time image of the worksurface andsuperimpose a virtualized line that visually separates one hundredgemstones from the rest of the gemstones 1650 thereby allowing theoperator to physically separate the one hundred gemstones from theremainder based on the image. For instance, FIG. 18 is an exemplaryscreenshot of the display of the output system 110 showing an image 1805of the gemstones 1650 on the worksurface 1685 and including asuperimposed dividing line 1810 separating a prescribed number ofgemstones from the remaining gemstones. Additionally, the foregoingsteps for imaging, image-processing and augmented display rendering canbe repeated in real-time such that, as the operator is separating outthe gemstones, the augmented display output is adapted accordingly. Forinstance, as the operator moves seventy of the one hundred gems to oneside of the worksurface, the output system 110 displays an updatedreal-time image of the worksurface with an updated dividing line so asto identify the remaining thirty gems for the operator to separate out.

As can be appreciated, when a large quantity of loose gemstones aredeposited onto a worksurface, they tend to disperse across the surfacenon-uniformly. More specifically, the gemstones can come to rest havingvarying orientations and resting positions (e.g., some are laying on aparticular side, some are resting on the top surface, etc.).Additionally, whereas some gems are spread apart, some gemstones can beclumped closely together, touching or even overlapping such that partsof some gemstone are partially obscured in the images. Additionally,gemstones can also be non-uniform in terms of size and shape and color,among other possibly varying physical characteristics. The foregoing canpresent a variety of practical challenges for automated systems forprocessing loose gemstones using image-based analysis techniques. Forexample, in practice, object detection algorithms such as a neuralnetwork can have difficulty in reliably differentiating betweengemstones that are touching or overlapping. This can lead tosub-standard performance when classifying, counting and processinggemstones based on images containing relatively large quantity of loosegemstones.

Accordingly, the systems and methods disclosed herein are specificallyadapted to overcome these and other challenges and provide an automatedsystem that can more accurately and effectively process a plurality ofloose gemstones using image-based analysis techniques.

According to a salient aspect, the processing unit 106 can be configuredto implement additional image-based processing and analysis steps (e.g.,in connection with one or more of steps 710, 715 and/or 720) to improvethe operation of the neural network or otherwise improve the ability ofthe processing unit to detect, differentiate, classify or otherwisemeasure characteristics of respective gemstones in an image depicting aplurality of gemstones. These and other analysis operations can beperformed on the image(s) captured of the gemstones, the diagramsgenerated therefrom (e.g., edge diagrams, point to point and line toline diagrams), or a combination of the foregoing. Additionally, in someembodiments, the analysis operations can be performed based at least inpart on the output of the neural network. Moreover, in some embodiments,the result of the additional image-based analysis operations can beprovided as an input to the neural network, prior to image processingusing the neural network and/or as feedback to the neural network.

In an embodiment, based on the detection of gemstones by the neuralnetwork at step 1715, the processing unit 106 can be configured toanalyze the image to determine physical characteristics of the group ofgemstones as well as individual gemstones detected by the neural networkat step 1720. As shown in FIG. 17, the result can inform furtherprocessing at 1725 and be provided as feedback to step 1715.

In an embodiment, the processing unit 106 is configured to analyze animage and/or associated edge diagram using a facet finding algorithm todetect gemstone facets within the image. Based on the detected facetsand respective edges, the processing unit can be configured to determinethe direction that respective facets extend. Additionally, theprocessing unit can further be configured to, based on the direction ofone or more adjacent or proximate facets, determine the orientation andresting position of a given gemstone. For clarity, the resting positionrefers to which portion of the gemstone is resting on the worksurface,such as, the gemstone's table, pavilion, crown, or culet and the like.The orientation of the gemstone can refer to, for example, the angle ofthe central axis of the gemstone relative to the plane of theworksurface.

Additionally, the processing unit 106 can further be configured todifferentiate between two abutting or overlapping gemstones based on thefacet directions. For instance, a set of one or more facets extending inone direction that are adjacent to a set of one or more facets extendingin a substantially different direction can indicate that the first setis associated with a first gemstone and a second set is associated witha second gemstone.

In an embodiment, the processing unit 106 can be configured to detectone or more key parts of respective gemstones from analyzing the imagesand/or diagrams. Key parts of a gemstone can include, a girdle, a table(e.g. top flat surface), a culet (e.g., bottom point), and one or morefacets. The processing unit can be configured to detect two parts of agiven gemstone based on relative position or proximity within the image.For instance, upon detection of a girdle within an image and facetsadjacent to the top and bottom edges of the girdle, the orientation ofthe gemstone can be determined based on the orientation of the girdleand angle of adjacent facets relative to the girdle. The processing unitcan similarly determine the orientation of a given gemstone based on therelative position of two positively identified key parts of a gemstone,e.g., table and culet, table and facets, culet and facets. Additionally,the size, shape and/or arrangement facets can be used to determine theshape of a gemstone.

It should also be understood that the methods for detecting key parts ofgemstones, identifying sets of key parts that are likely to beassociated with the same gemstone, determining a gemstone's position ororientation based on the relative position and orientation of key partscan be informed by one or more models representing the expectedrelationship of key parts for each of a plurality of known gemstoneshapes/cuts (e.g., round, oval, princess, cushion, etc.). The models canalso specify the expected relationship for each gemstone cut as afunction of possible resting positions and orientations on aworksurface.

Similarly, the processor can be configured to determine the cut of agiven gemstone in the image by comparing the relative position andorientation of key parts of a given gemstone to the models representingthe structural arrangement of key parts for each of the plurality ofknown gemstone cuts and for each of a plurality of possible restingpositions and orientations.

In addition to determining the shape/cut of a gemstone detected in animage, the processing unit 106 can be further configured to determineother physical characteristics of individual gemstones in the imageincluding, for example, the approximate size of each gemstone. Morespecifically, in an embodiment, the processing unit 106 can beconfigured to determine the curvature of a girdle of a given gemstonefrom the one or more images. Based on the curvature of the girdle, anapproximation of the girdle's circumference and thus the gemstone's size(e.g., volume and carat weight) can be calculated. The calculation of agemstone's size can also be informed by the particular cut of thegemstone and other approximated dimensions measured from the image(s)for example, the distance between key parts such as the table and girdleand culet. Size can also be determined based on angles of key parts.

In some embodiments, the processing unit 106 can be configured todetermine the curvature of a girdle by analyzing the shape of a givengemstone's girdle as depicted in each a first image of the gemstones,which is taken at a first known angle relative to the worksurface, and asecond image of the gemstones taken from a different angle. Preferably,images are captured of the gemstones from at least two different anglesto provide sufficient information concerning the 3-dimensional structureof each gemstone (e.g., curvature and shape) to enable the processingunit to estimate the structure and volume of the gemstones respectively.

Although weight estimation from a few images depicting multiple loosegemstones resting in non-uniform positions can result in estimationsthat are above or below the actual weights, such inaccuracies tend toaverage out, particularly when the system 1600 is used for processingmultiple gemstones at a time, say, separating out twenty-five gemstoneshaving a round cut and average weight of one carat.

In an embodiment, the two or more images can also be used by theprocessing unit 106 to recreate approximate structure of respectivegemstones within the image of the gemstones and generate a diagram forindividual gemstones. As noted above, diagrams of individual gemstonescan be in the form of point-to-point and line-to-line diagramsrepresenting the overall structure of the gemstone.

In some embodiments, the processing unit 106 can be configured todifferentiate/distinguish touching or overlapping gemstones based onmultiple images of the gemstones 1650 captured from multiple differentangles. For example, gemstone detection at step 1715, can be performedon each image and the results combined. By way of further example,results can be compared to identify any discrepancies or irregularobjects that might require further analysis (e.g., at step 1720) usingone or more of the gemstone analysis, feature detection, differentiationand measurement techniques described above.

Additionally, in some embodiments, partial views of the gemstonesdepicted in different images captured at different angles can becombined and used to reconstruct a more complete two-dimensional orthree-dimensional image or diagram of the gemstones in accordance withthe systems and methods disclosed above. Furthermore, the compositeimages or diagrams can similarly be analyzed to detect and evaluatepartially obstructed gemstones.

In some embodiments, the processing unit 106 can also be configured tomeasure a grade for the cut (“cut grade”) based of the detected edges,facets, girdle, gem structure from either one or more sides of theobject. The gemstone can also be configured to generate a diagram thatrepresents the ratios and placements of the parts (like facets andgirdle etc.) that makes up the entire structure. Determining a cut gradecan also be based on a model of what the “best” cut of a gem or angle ofa prong or curvature of a jewelry item should be, which could be basedon aesthetics, mathematics, golden ratio, Phi or any given model againstwhich to base off the grading.

In some embodiments, the image capturing device 102 can further includeone or more distance or depth measuring sensors, such as LIDAR—laserdistance and ranging devices, sonar sensors or other suitable distanceor depth measuring devices useable to measure the distance of a target(or portion thereof) from the capturing device 102 and thus measurethree-dimensional structure of the target. Based on the measureddistance between the image capturing device and gemstones (or partsthereof) being imaged and the focal settings of the camera, theprocessing unit can be configured to measure a size of thegemstones/parts within the image as well as determine itsthree-dimensional structure.

In some embodiments, the processing unit 106 can be configured tomeasure the size and shape of the gemstones using depth from focustechniques. For instance, the image capturing device 102 can beconfigured to capture a plurality of images while sweeping the focus ofthe camera through a range of focal settings. The processing unit 106can be configured to analyze the imagery to identify which portions ofone or more gemstones are in focus in respective images and, based onthe corresponding focal setting, measure size, shape among otherthree-dimensional structural characteristics (e.g., curvature etc.) ofthe one or more gemstones. By way of further example, a trained neuralnetwork can be used to analyze the images to visually measure distancefrom the imagery and/or differences in the focus point and sharpness ofareas within the image to find distance or length, width and depth ofthe imaged objects.

By way of further example, the processing unit 106 can be configured touse the image capturing device 102 to capture images that are eachfocused on a different part of a gemstone (e.g., the culet, girdle,table, edge, etc.) and analyze the images using a neural network trainedto measure distance based on the in-focus and out of focus portions ofthe image and corresponding focus settings.

In some embodiments, the worksurface can be provided with one or morereference markings such as lines, dots, shapes, letters, numbers and thelike to facilitate the image processing, gemstone analysis and gemstoneprocessing operations. The reference markings can be arranged in two orthree dimensions and have a prescribed size and spatial relationship.Based on the prescribed spatial relationship of the reference lines,which can be input to the processing unit 106, and a known distance andangle of the image capturing device relative to the worksurface duringimage capture, the weight of one or more gemstones can be determinedbased on a comparison of the size of the gemstone relative to one ormore of the reference lines. Additionally, in some embodiments, theprescribed location of one or more reference markings can be used todetermine the position of each gemstone on the worksurface. In someembodiments, the reference markings can further include encodedinstructions that are machine readable and useable to provideinstructions to the processing unit 106 or other such computer-visioncontrolled devices.

Furthermore, in some embodiments, the processing unit 106 can beconfigured to measure the color of respective gemstones. As can beunderstood, the physical characteristics respectively measured by theprocessing unit 106 for the gemstones can be recorded in memory. Forexample, the position, size, shape, weight, location and color of eachgemstone on the work surface can be recorded.

Turning to FIG. 19, an exemplary worksurface 1905 is shown that can beutilized with the system 1600 to facilitate the techniques describedherein. The dimensions of a gemstone among a plurality of gemstones 1950and/or features of the gemstone such as a size of a girdle, facets andthe like can be determined with reference to the worksurface, which canhave one or more reference lines of known thickness and length and/orspacing printed thereon. For example, the worksurface 1804 can havelines 1920-1928 and 1930-1944 arranged in a grid. The spacing of thelines in the grid can assist with determining a size and/or shape of thegemstones. Additionally, the grid can represent a coordinate system usedby the processing unit to determine or otherwise assign a respectivelocation to each gemstone in an image.

Analyzing a given gemstone in view of the lines can include processingthe captured image to determine one or more dimensions of a givengemstone by comparing image data corresponding to the gemstone to one ormore of the lines 1615-1625. In certain embodiments, the worksurface1604 and/or the lines printed thereon are also or alternatively used todetermine one or more shapes of the gemstone 1602 and/or quantify one ormore of the aforementioned characteristics of facets, a girdle or otherparts of the gemstone.

As noted, in some embodiments, reference markings can be provided inthree-dimensions. For instance, although not shown in FIG. 19, one ormore side-walls extending vertically (e.g., in the Z direction) fromedges of the two-dimensional worksurface 1804 (which extends in the X-Yplane) can be provided with similar reference markings and can be usedto determine a height of gemstones in the Z-direction. In anotherembodiment, the worksurface comprises a gemholder that is a box-likestructure.

In certain embodiments, the worksurface can comprise a base that iscovered by a piece of paper, a plastic sheet, or other suitable materialon which reference markings (e.g., lines, dots, etc.) are printed andgemstones are placed. In some embodiments, the worksurface istransparent, such as a glass substrate, or translucent such that thegemstones can be illuminated by a light source 180 provided below theworksurface.

In some embodiments, reference markings having prescribed colors canalso be provided on the worksurface, e.g., green, red and blue referencemarkings. Based on a prescribed hue of the color reference markings, theprocessing unit 106 can color correct the image and determine the colorof respective gemstones.

While a grid pattern is shown in FIG. 19, it should be understood thatother types of reference markings and other arrangements of markings canbe used to determine gemstone shape, location and orientation, among orother salient physical properties.

In some embodiments, one or more lines can be provided on theworksurface to assist with processing operations such as sorting. Forinstance, as shown in FIG. 19, the central vertical line 1923 can definea side of the worksurface (e.g., left-side) that gemstones are moved toduring a sorting or separating operation (e.g., as performed at step1725).

One or more of the exemplary physical characteristics of the gemstonesdetermined by the processing unit 106 can be used to perform variousgemstone processing operations at step 1725. As noted above, in oneexample, the system 1600 can be configured to separate out gemstonesbased solely on gemstone count and without regard to other physicalcharacteristics and user specified parameters/criteria. In a moreadvanced example, the processing unit 106 can be configured to select aset of gemstones from the assortment of loose gemstones 1650 accordingto a combination of parameters, such as, a specified number of gemstoneshaving a prescribed total weight and wherein each gemstone has aparticular cut/shape and color and a minimum individual weight (e.g., 1carat minimum).

In an embodiment, the processing unit 106 can be configured toautomatically assist with selection of the subset of gemstones byoutputting a real-time image of the gemstones 1650 augmented by markingshighlighting the gemstones in the set meeting the prescribed criteria.Furthermore, as individual gemstones are selected by the operator, theprocessing unit can be configured to update the set of gemstones and theaugmented display accordingly.

In a further embodiment, the processing unit 106 can facilitateautomated gemstone processing at step 1725, for instance, by sendinginstructions that cause an automated robot to perform the selection andsorting operations. More specifically, the processing unit 106 can beconfigured to transmit an instruction to a robot via a communicationinterface (e.g., the output system 110). The instruction can specify arespective location of a plurality of individual gemstones that theprocessing unit identified as having a prescribed physicalcharacteristic. Additionally, the instruction causes the robot tophysically move the plurality of individual gemstones and/or performother possible processing operations. For instance, the processing unit106 can instruct a robotic arm to pick up the set of gemstones byspecifying the respective coordinate of the gemstones (e.g., as definedby the reference lines/points).

It should be understood that counting and selecting gemstones from amonga large quantity of assorted loose gemstones are just a few examples ofpossible operations that can be performed using the system 1600 based onthe image-based analysis techniques described above. The system 1600 canbe configured to facilitate the performance of a variety of othergemstone processing operations using robots and/or manual operators.

Additionally, although the foregoing discussion of the system 1600 hasbeen described in the context of processing a plurality of loosegemstones, the system can similarly be used to process a plurality ofgemstones that are held by a gemholder or a jewelry item in which thegemstones are set in prongs. For instance, a bracelet having a pluralityof gemstones can be laid on the worksurface and the system 1600 canperform one or more steps of the method 1700 accordingly. For example,the number of gemstones can be counted and characteristics evaluated.Additionally, automated processing operations can be performed by thesystem using, for example, a robot, such as, opening prongs holdingcertain gemstones, removing the selected gemstones, re-setting thegemstones and the like. By way of further example, the system can beconfigured to identify missing gemstones from the jewelry item, andsuggest a suitable replacement (e.g., based on size, color etc. of theother gemstones), automatically identify a suitable replacement from aset of loose gemstones and set the replacement gemstone into the jewelryitem.

It should be understood that although a single gemstone processingsystem 1600 has been shown and described as processing a set ofgemstones on a single worksurface, it should be understood that, in someembodiments, multiple such systems can be networked and configured tocommunicate so as to perform gemstone processing operations jointly, asan integrated system, and/or independently.

As noted, the system 1600 comprises an input system 108, such as atouchscreen input device by which an operator can provide inputs andinstructions to the processing unit 106 that control or otherwise guideoperation of the system 1600. For example, an operator viewing thereal-time image of the gemstones on the workspace 1685 can tap thetouchscreen to select particular gemstones, say, specific gemstones theoperator desires to be included in the set of gemstones processed atstep 1725.

In some embodiments, the system 1600 can be configured to receive userinputs via the images (e.g., using computer-vision techniques) andadaptively perform one or more of the various gemstone analysis andprocessing operations discussed above accordingly. For instance, theprocessing unit 106 can be configured to analyze the images captured bythe camera 102 to detect whether they depict the user performing one ormore of a prescribed set of gestures that are stored in memory andassociated with respective processing operations that the system ispre-programmed to perform. For example, a user pointing a tweezer orother pointing device at a particular gemstone can be interpreted by theprocessing unit 106 as a selection of that particular gemstone forfurther analysis (e.g., at step 1715 or 1720) or processing (e.g., atstep 1725). By way of further example, a user tapping the particulargemstone a prescribed number of times (e.g., once, twice or three timesetc.) is interpreted as an instruction to perform a correspondingoperation on that gemstone. By way of further example, the processor canbe configured to interpret the pointing of a tweezer at an object or anarea on the workspace as an identification of a “point of action” andcauses the processing unit 106 to display a set of selectableinstructions on an interactive real-time display. Additionally, theprocessing unit can be configured to receive and act upon voice-commandsor other types of commands.

Additionally, in some embodiments, the processor can be configured toanalyze markings provided by the user on the worksurface to identifyinstructions or other such commands. For instance, a user drawing a linearound an area can be interpreted as specifying that area as a point ofaction and to perform processing operations on the gemstones within thearea. It should be understood that the processing unit can be configuredto receive similar user inputs via a touch-screen display.

By way of further example, instructions can be provided on or within theworksurface (either manually drawn, pre-printed, etched into or embeddedwithin) comprising a diagram including points of action with respectivemachine readable instruction codes that instruct an automated gemstoneprocessing robot to perform a prescribed operation at the respectivepoint of action. Although the system 1600 comprises one or more elementssuch as the processing unit 106 and image capturing device 102 providedseparately from the output device such as an automated gemstoneprocessing robot, one or more of these components can be integrated intothe gemstone processing robot. In some embodiments, a processing unitand image capturing device can be provided on-board the automatedgemstone processing robot thereby enabling the gemstone processing robotto perform one or more of the foregoing operations using computer-visiontechniques.

It should be understood that various combination, alternatives andmodifications of the disclosure could be devised by those skilled in theart. The disclosure is intended to embrace all such alternatives,modifications and variances that fall within the scope of the appendedclaims.

It is to be understood that like numerals in the drawings represent likeelements through the several figures, and that not all components and/orsteps described and illustrated with reference to the figures arerequired for all embodiments or arrangements.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the disclosure.As used herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising”, when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

Also, the phraseology and terminology used herein is for the purpose ofdescription and should not be regarded as limiting. The use of“including,” “comprising,” or “having,” “containing,” “involving,” andvariations thereof herein, is meant to encompass the items listedthereafter and equivalents thereof as well as additional items.

The subject matter described above is provided by way of illustrationonly and should not be construed as limiting. Various modifications andchanges can be made to the subject matter described herein withoutfollowing the example embodiments and applications illustrated anddescribed, and without departing from the true spirit and scope of theinvention encompassed by the present disclosure, which is defined by theset of recitations in the following claims and by structures andfunctions or steps which are equivalent to these recitations.

1. A method for processing a plurality of loose gemstones resting on aworksurface, the method comprising: receiving, by a computing devicehaving a non-transitory computer-readable storage medium and a processorconfigured by executing a software program stored in the storage medium,from an image capturing device, an image of the plurality of loosegemstones on the worksurface; detecting, by the processor using atrained neural network, individual gemstones within the image;performing, by the processor based on the detecting step, a processingoperation on the gemstones.
 2. The method of claim 1, furthercomprising: detecting, by the processor based on the image, a physicalcharacteristic of the individual gemstones respectively; and wherein theprocessing operation is performed as a function of the respectivelydetected physical characteristics.
 3. The method of claim 1, whereinperforming the processing operation comprises: receiving, by theprocessor via a user interface, a parameter according to which one ormore individual gemstones are to be physically separated from theplurality of loose gemstones; identifying, by the processor based on theparameter, at least one individual gemstone in the image that satisfiesthe parameter; outputting, by the processor via an associated displaydevice, the image of the plurality of gemstones; and rendering, by theprocessor on the displayed image, one or more superimposed markings inrelation to the identified at least one individual gemstones.
 4. Themethod of claim 1, further comprising: detecting, by the processor basedon the image, touching gemstones depicted in the image; anddifferentiating between individual gemstones among the touchinggemstones.
 5. The method of claim 4, wherein the step of detectingtouching gemstones includes: detecting gemstone features depicted withinthe image using a feature detection algorithm; detecting the touchinggemstones based on relative positions of the detected features; anddifferentiating between the individual gemstones among the touchinggemstones as a function of the detected features.
 6. The method of claim5, wherein the step of detecting individual gemstones using the neuralnetwork is performed as a function of a result of the differentiatingstep.
 7. The method of claim 1, further comprising: detecting, by theprocessor based on the image, at least two parts of each individualgemstone respectively, the at least two parts being selected from thegroup consisting of a surface a gemstone feature, a facet, a girdle, aculet and a table; measuring, by the processor as a function of arelative position of the detected at least two parts, a characteristicof the individual gemstones respectively, wherein the characteristic isselected from the group consisting of, an orientation, a restingposition, a shape, and a size; and wherein the processing operation isperformed as a function of the respectively measured characteristics. 8.The method of claim 2, wherein the worksurface is provided with one ormore reference markers, wherein the step of detecting the individualgemstones includes determining respective locations for the individualgemstones as a function of the one or more reference markers, andwherein the measuring step is performed based on the one or morereference markers.
 9. The method of claim 1, wherein performing theprocessing operation comprises: transmitting via an output interface toa robot in communication therewith, an instruction specifying arespective location of a plurality of individual gemstones having aprescribed physical characteristic and wherein the instruction causesthe robot to perform the processing operation on the plurality ofindividual gemstones.
 10. A system for processing a plurality of loosegemstones resting on a worksurface, the system comprising: an imagecapturing device for capturing an image of the plurality of loosegemstones on the worksurface; a computing device having a non-transitorycomputer-readable storage medium and a processor, wherein the processoris configured by executing a software application to: detect, based onthe image using a trained neural network, individual gemstones withinthe image, and perform, based on detecting the individual gemstones andusing an output device, a processing operation on the gemstones.
 11. Thesystem of claim 10, wherein the processor is further configured to:detecting, based on the image, a physical characteristic of theindividual gemstones respectively; and wherein the processing operationis performed as a function of the physical characteristic.
 12. Thesystem of claim 10, further comprising: a user interface in operativecommunication with the processor, wherein the processor is configured toreceive, via the user interface, a parameter according to which one ormore individual gemstones among the plurality of loose gemstones are tobe processed; the output device comprising a display device in operativecommunication with the processor; and wherein the processor isconfigured to perform the processing operation by: identifying, based onthe parameter, a plurality of individual gemstones in the image thatsatisfy the parameter, outputting, by the processor via the displaydevice, the image of the plurality of gemstones, and rendering on thedisplayed image, one or more superimposed markings in relation to theidentified at least one individual gemstones.
 13. The system of claim10, wherein the processor is further configured to: detect, based on theimage, touching gemstones depicted in the image and differentiatebetween individual gemstones among the touching gemstones.
 14. Thesystem of claim 13, wherein the neural network is trained to detect theindividual gemstones based on the processor differentiating betweenindividual gemstones among touching gemstones.
 15. The system of claim10, wherein the processor is further configured to: detect, based on theimage, at least two parts of each individual gemstone respectively, theat least two parts being selected from a group consisting of a surface,a gemstone feature, a facet, a girdle, a culet and a table; and measure,as a function of a relative position of the detected at least two parts,a characteristic of the individual gemstones respectively, wherein thecharacteristic is selected from the group consisting of, an orientation,a resting position, a shape, and a size; and wherein the processorperforms the processing operation as a function of the respectivelymeasured physical characteristics.
 16. The system of claim 10, furthercomprising: the worksurface, wherein the worksurface includes one ormore reference markers, and wherein the processor is configured todetermine respective locations for the individual gemstones as afunction of the one or more reference markers, and wherein the processorperforms the processing operation based on the respective locations forthe individual gemstones and the one or more parameters.
 17. The systemof claim 10, wherein the output device comprises a communicationinterface and wherein the processor is configured to perform theprocessing operation by: transmitting an instruction via thecommunication interface to an automated gemstone processing robot, theinstruction specifying the respective locations of a plurality ofindividual gemstones having a prescribed physical characteristic andwherein the instruction causes the robot to physically perform theprocessing operation on the plurality of individual gemstones.
 18. Themethod of claim 1, wherein the detecting step includes: analyzing, bythe processor, the image to detect edges of objects within the image;and generating a diagram representing the detected edges of thegemstones, wherein the neural network detects the individual gemstoneswithin the image based on the diagram.
 19. The method of claim 1,further comprising, counting the individual gemstones detected withinthe image to determine a gemstone count, and wherein the processingoperation is performed as a function of the gemstone count.
 20. Themethod of claim 1, wherein the step of detecting a physicalcharacteristic further comprises: illuminating, while capturing theimage, the plurality of gemstones using one or more light sourcesconfigured to cause the physical characteristic to stand out in theimage.
 21. The method of claim 1, wherein the image comprises aplurality of images, and wherein each image among the plurality ofimages is captured from a respective angle relative to the worksurface.22. The method of claim 2, wherein the wherein the physicalcharacteristic is selected from a group consisting of a shape, a weight,a size, a surface feature, an internal feature, an inclusion, and acolor.